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ABSTRACT 

This is a report of a study that examined the statistical arid 
institutional influences on the prediction of first-year college 
grades: The husic sources of information were the Validity 
Study Services file which summarizes the results of College 
Board validity s:adies and the College Handbook tile which 
includes data abcut college characteristics. The criterion 
was the size of the multiple correlation between academic 
predictors and Hrst-year college grades. The independent 
variables were the statistical data of the vaJ dity study and 
college characteristics. In general it was found, as expected, 
that the extent of the variation of the academic ability of the 
students was positively related to the size of the multiple 
correlation, In addition, several variables suggested the 
interpretation that ihe heterogeneity of the programs arid 
experience of college were negatively related to the size of 
the multiple correlation. 

Further analyses investigated the characteristics that 
were associated with the greater or lesser efficiency of s.T- 
verbai scores and sat- mathematical scores in the predicti , 
of grades. Similar analyses examined the characteristics 
associated with greater or lesser efficiency of sat scores and 
the high school record: Finally, the prediction of grades at 
colleges of different selectivity was examined: In all of 
these analyses evidence was provided for the influence of 
institutional characteristics on the prediction of grades, an 
influence that needs to be taken into account when 
interpret ihg the "validity" of tests like the sat. 



INTRODUCTION 

The validity of the Scholastic Aptitude Test (sat) and high 
school grades — the correlation between these variables arid 
college grades — has long been a concern of Users of the 
sat, the College Board and Educational Testing Service, 
and critics of standardized testing: The college user of the 
sat and grade records is chiefly concerned with the validity 
of the test and grades at his or her own campus: The College 
Board and Educational Testing Service look at the validity 
of the sat as one of the criteria of the quality of the 
instrument. Finally, one of the criticisms of test critics is 
that the sat does not adequately predict giades. 

Each of these groups generally thinks in terms of the 
prediction of individual grades, and a great deal of rese^xh 
has been done on the predictability of an individuals grades 
arid the personal characteristics that are related to predic- 
tion. This research has included bver : arid u rider-predict ion, 
under- arid over-achievement, moderator variables, arid so 
bri. This large literature has also focused bri the validity of 
tests among different populations (Brelarid 1978), the 
difficulty of long-term prediction (Humphreys 1968; Wilsbri 
1983), the reasons, such as the diversity of grades, that 
grades are difficult to predict (Goldman and Slaughter 
1976), and most recently, the qaestion of validity generaliza- 



tion: However, most of the research on the prediction of 
college grades has neglected the imnnmonal characteristics 
that influence the predictability of college grades: For 
example; on both statistical and logical grounds; the range 
of sat scores and high school grades in a class should have a 
marked effect on the predictability of college grades. Linn 
( 1980) has demonstrated that using data from Schrader 
(1977), the size of the correlation between lsat scores and 
first -year law school grades is appreciably influenced by the 
size of the standard deviation of the lsat scores of a schools 
students. 

The purpose of the research reported here is to examine 
the influence of a variety of institutional arid student body 
characteristics bri the predicti bri of college grades. Some of 
these influences are quite Understandable. For example, 
Rarriist (1980) has shown that the size of the sat- fresh rri an 
grade correlation is affected by the size of the sAT-standard 
deviation: Among colleges with standard deviations of 100 
or higher, the mean correlation is :50; among colleges with 
standard deviations below 80, it is :36: This resalt woald be 
expected on statistical grounds; it also would be expected on 
logical grounds: the greater the range of academic talent 
among the students entering an institution, the greater the 
range of grades that might be expected to be awarded, and 
the greater the potential role of academic ability in 
determining grades. Thus, whatever increases the hetero- 
geneity of academic input is likely to increase the 
predictability of grades. Similar arguments could be made 
about the relative power of predictors to predict college 
grades, as Dawes (1975) has argued. For example, if a 
college is very selective bri SAT-mathematical scores, the 
verbal scores may be the only predictor with sufficient 
variance to correlate with college grades. 

Other research has shown that th: predictability of 
grades is also affected by the heterogeneity of the academic 
programs of colleges. That is, the more diverse the curricula 
in terms of major fields, divisions, colleges, and so forth, 
the less predictable are grades (Monday 1970): In addition, 
the less homogeneous the college experience* the less 
predictable the grades. For example, grades at urban 
colleges enrolling many part-time and working students 
might be less predictable simply because these students' 
academic performance can be affected by many influences 
other than the institution's program. Munday (1970) found 
that grades at colleges with students living «ri college- 
controlled residences were more predictable than grades at 
colleges with commuting students, arid Rarriist (1980), after 
controlling for the size of the test standard deviation, fbUrid 
that grades in small colleges were more predictable than 
those in large colleges. It might also be expected that 
colleges with core curricula would have higher correlations 
than colleges with completely elective systems. It might 
further be expected, for example, that at colleges emphasiz- 
ing mathematics and science in their curricula, ;he sat- 
mathematical score might be a better predictor than the sat- 
verbal. 
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Some characteristics of colleges that have been found 
to be related to the prediction of grades do not have very 
clear-cut explanations; For example; some studies _have 
shown that the grades of four- year college students are more 
predictable than those of community college students; the 
grades of students in private colleges more predictable than 
those in public colleges, and the grades of students in high- 
cost institutions more predictable than those in low-cost 
institutions. Although it is possible to speculate as to the 
reasons for these differences, the differences will simply be 
noted as factors related to predictability. 

Finally, and again expectedly on theoretical arid logical 
grounds, the range of the criteria is related to predictability. 
Not drily is the standard deviation of the grades awarded 
related, but also such factors as the percentage of pass/fail 
courses allowed students, policies on dropping courses, 
whether low grades can be expunged from the academic 
record oy repeating a course, and so on. That is; the smaller 
the range of grades awarded and the less homogeneous and 
reliable the basis upon which grade averages are based, the 
lower their predictability: (Furthermore, Breiand [1979] and 
Werts, Linn; and Joreskog [1978] report single year gpa 
reliability coefficients of about .60, which clearly limit the 
size of any correlation.) 

In sum, the size of the correlation between admissions 
tests such as the sat and grades is influenced by many 
factors that have nothing to do with the intrinsic validity of 
the tests themselves. Some of these factors, such as the 
range of scores, have clear statistical explanations; some, 
such as the percentage of students in college-controlled 
housing, have sociological explanations; arid some, such as 
the percentage of women in the class, do riot have clear 
explanations although they can have a marked effect on 
predictability. Whatever their underlying mechanism, these 
factors can appreciably increase or reduce the size of the 
correlation between test scores and grades: 

In the current discussion about the validity of tests, it 
would be useful to demonstrate how the validity of the 
tests — the correlation between test scores and freshman 
grades — c?n be influenced by various factors. The point is 
not to argtu that the correlation in the most predictable 
colleges is the "rear 1 one, but that correlations must be 
interpreted within the context of the characteristics of the 
institution. These characteristics put limits on the s[ze of the 
correlations that are possible arid expectable. There are 
conditions in which the correlation could not be expected to 
be high. Obviously, each correlation must be evaluated in 
terms of local conditions. Also, it is clear that it does riot 
make much sense to use an 4 4 average" correlation 
coefficient as an indicator of the expected validity of a test; 
particularly since some of the very characteristics that are 
associated with participating in the validity service; such as 
.selectivity, also may be associated with lower validity 
coefficients: Thus; this study attempted to demonstrate the 
influence of institutional characteristics on the obtained and 
expectable correlations between SAT scores and grades, and 



to examine the pattern of validity results as they relate to 
these characteristics. 

METHODS 

The basic data for this study were obtained from the College 
Board's Validity Study Service (vss) files for 1981 arid the 
files for the Hie College Handbook for 1981. 

The vss files provide information about the following: 

1 . Number of students in the study 

2. Mean sat- verbal arid sat- mathematical scores 

3. High school record, reported in four different 
ways: (a) college reported high school grades; (b) 
college reported high school rank; (c) student 
reported high school average; arid (d) student 
reported class rank 

4. Standard deviations for sat- verbal arid sat- 
mathematical scores 

5. Standard deviations for the four high-school- 
grade-performance measures 

6. The mean college gpa (cgpa) earned by the 
students in each study 

7. The standard deviation of the cgpa in each study 
8: The multiple correlations of the two sat scores 

with CGPA 

9. The multiple correlations of the two sat scores 
and high school grades with cgpa 

The Coiiege Handbook file includes information about 
the following: 

10. Full-time undergraduate enrollment 
H. Part-time undergraduate enrollment 

12. Total Undergraduate enrollment 

13. Total graduate enrollment 

14. Minimum gpa for good standing 

15. Percentage of freshmen in good standing 
i6: Percentage of transfers in good standing 

17 . Number of men who applied to the freshman class 
18: Number of n~en who were accepted as freshmen 
19; Number of men who enrolled as freshmen 

20. Number of women who applied to the freshman 
class 

21 . Number of women accepted as freshmen 

22. Number of women who enrolled as freshmen 

23. Percentage of students from within the state 

24. Percentage of students who live in college housing 

25. Percentage of students who live at home arid 
commute 

26. Percentage who live in private housing 

27. Percentage of all students who commute 

28. Percentage of students who are minority students 

29. Percentage of students who are foreign 

30: Percentage of students majoring in architecture 
31. Percentage of students majoring in liberal arts 
32: Percentage of students majoring in business 
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33. Percentage of students majoring in education 

34. Percentage of students majoring in engineering 

35. Percentage of students majoring in arts 

36. Percentage of students in health sciences 

37. Percentage of students in home economics 

38. Percentage of students in religion 

39. Percentage of students in technology 

40. Percentage of students who transferred from two- 
year colleges 

41. Percentage of students who transferred from four- 
year colleges 

42. Percentage of graduates of two-year programs who 
enter four-year programs 

43. Percentage ot graduates of four-year programs 
who enter graduate programs 

44; Number of communication and publication 

activities available 
45; Number of performing arts activities available 

46. Number of sports activities available 

47. Tuition and fees 

48. Tuition and fees and room and board 

49. Number of freshmen receiving financial aid 

50. Percentage of freshmen who are offered full 
financial aid 

The previous variables are ail continuous variables. In 
addition, a number of categorical variables from The 
College Handbook file were included in the analyses: 

5 1 . College level: two-year or four-year 

52. Public or private college 

53. Liberal arts college or university 
54: Catholic or Protestant college 

56: Type of academic calendar: semester, quarter, 
trimester, or 4-1-4 

57. Location: metropolis, large city, small city, 
suburb, large town, small town, rural 

58. Offers A A degree or does not 

59. Offers BA degree or does not 

60. Offers academic acceleration or does hot 

61. Offers honors or does not 

62. Allows student to design major or does hot 

63. Allows double major or does hot 

64. Offers independent study or does hot 

65. Provides internships or does not 

66. Provides cooperative education or does not 

67. Offers study abroad or does not 

68: Offers combination program of libera! arts and 

career program or does not 
69: Provides remedial courses or does not 

70. Provides tutoring or no tutoring 

71 . Provides counseling or no counseling 

72. Has a learning center or does not 

73. Gives credit to AP scores greater than 3, versus 
giving credit to lower scores 

74- Allows clep credits or does hoi 

75. Requires a minimum gpa versus no minimum 



76. Requires interview for admission or no interview 

77. Open admission poiicy or rib open admission.. 

78. Application deadline: January-April, May-Sep- 
tember, rib closing date 

79. Has fraternities arid/br sororities, or has neither 

Statistics 

Because some institutions conduct riiUltiple validity studies, 
whereas others conduct only one, there are two Ways io 
examine this data. The first is to use the validity study as the 
unit of analysis. This approach concentrates on the variables 
that affect the results of the study: The second is to use the 
institution as the unit of analysis: This approach concen- 
trates on identifying the types of institutions that tend to 
have higher or lower validity coefficients. Both types of 
analyses were conducted in this study. The institutional 
sample was constructed by identifying the most representa- 
tive study conducted by each institution and using it with 
other data about the institution. 

Because the results of multiple regression analyses are 
affected by the number of predictors, studies were also 
chosen for the analyses that had only the three basic 
predictors, sat- verbal scores, sat- mathematical scores, arid 
high school grades. In addition, only L io.se studies that 
included all freshmen rather than subgroups were included. 
Finally, the studies that appeared to be ate mast representa- 
tive of the total freshman classes were included: 

Variables j through 9 were correlated, asing the validity 
study as the unit of analysis for examining their relation- 
ships: The two multiple correlations (SAT-verbal and sat- 
mathematical with grades, and SAT-verbal and sat- 
mathematical plus the high school record) were considered 
the dependent variables; the other variables were considered 
independent variables. These analyses were repeated for 
variables 1 through 50 using the institution as the unit of 
analysis. The mean multiple correlations for each categori- 
cal variable (5 1-7?) were also calculated using the validity 
study as the unit of analysis. A difference of .03 in the mean 
multiple correlation was considered important enough to 
note. 

Further analyses were conducted using the multiple 
correlations as the dependent variables. Multiple regression 
was used to study the relationship between the other 
variables and the size of the multiple correlations to predict 
freshmen grades: 

Additional analyses examined the question of the relative 
effectiveness of the various predictors in predicting college 
grades. This was done by creating a variable which was the 
difference between the correlation of SAT-verbal scores with 
college grades (t VC cpa) and the correlation of sat- mathe- 
matical scbres_ with college grades Xtmcgta), or 
(Wccpa ~ Tmcgp.x)' Then this variable was correlated with the 
other statistical arid irisiitutiorial variables. That is, the 
analyses were designed to address the question as to why V 
is a relatively better predictor in some colleges, whereas M 
is a better predictor in others. 
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Similar analyses were conducted to examine the possible 
reasons for the relative effectiveness of SAT scores ahdjhe 
high school record in predicting college grades. Two 
variables were created, subtracting the correlation of verbal 
scores from the correlation of the high school record 
(TmcpAccpA ~ "vcgpa). and subtracting the con elation of 
mathematical scores from the correlation of the high school 
record \Jhsgp\cgpa cg~pa)' These variables were then 
correlated with the statistical and institutional variables. 

Finally, because of the widely held belief that the 
prediction of grades is inherently more difficult at selective 
institutions, analyses of the patterns of validity results at 
colleges with different mean combined verbal and mathe- 
matical scores were conducted: 



RESULTS 

Table 1 shows the correlations between the statistical 
characteristics of the students at each college and the size of 
the two mui^ple correlations: (1) sat- verbal scores and sat- 
mathematical scores with college grades (R 2 )\ and (2) sat- 
verbal scores, SAT-mathematical scores; and high school 
grades with college grades (/?,)• The first two variables, 
reflecting the average level of ability of the students in the 
college — mean SAT-verbal and SAT-mathematical scores — 
were not meaningfully related to the prediction of grades. 
This result does not confirm Munday s (1970) finding that 
the level of ability was positively related to the prediction of 
grades. However, the range of academic ability as reflected 
in the standard deviation of the sat scores of the students in 
the college was related to both multiple cbitelatibris. 

Although the standard deviation of the college grades 
obtained in each study was expected to have a substantial 
influence on the multiple correlations, it did hot. The mean 
college grade awarded across the studies was also unrelated 
to the size of the multiple correlations. 

Table 2. Correla tions Between Institutional Ch 



Enrollment Variables 

Full time undergraduate enrollment 

Total undergraduate enrollment 

Total enrollment 
Residence Variables 

tercentage who live at home and commute 

Percentage living in college housing 
Minority Students 

Percentage of enrollment that Is mino- :t y 
Activity Variables 

Number of communication and publishing activities available 

Number of sports activities available 
Aid Jeered 

Percentage of freshmen in financial need 

Percentage of freshmen offered full financial aid 

♦This correlation is significant at the .05 level. 
**This correlation is significant at the .01 level. 



Table 1. Correlations with Multiple Correlations 
Pn3ictjng_ C ollege Grade s 

Correlation with: 





Rft 


N 


R,tt 


N 




SAT-V mean 




— .08 


i c< 

1 J7I 




— .03 


i n 

i / / 


SAT-M mean 


-:I3 


155 


-:07 


177 


SAT-V standard deviation 


.31** 


155 


.25** 


177 


SAT-M standard deviation 


.30** 


155 


.29** 


177 


College GPA mean 


10 


150 


.13 


172 


Coliege GPA standard deviation 


-.10 


150 


-.04 


172 


HS rank mean 


- 21* 


85 


-.18 


97 


HS GPA mean 


16 


70 


11 


80 


HS rank standard deviation 


.14 


85 


20* 


97 


HS GPA standard deviation 


-.13 


70 


-.13 


80 



Note: Only studies that used high school GPA and college grades on a 0 to 
4 scale were used. 
tRj is the multiple correlation of SAT-verbal and SAT-mathematical 
scores with college grades. 
ttR, is the multiple correlation of SAT- verbal scores. SAT- mathematical 
scores, and high school grades with college grades. 
♦This correlation is significant at the .05 level. 
**This correlation is significant at the 01 level. 

High school grades can be reported as ranks or average 
grades in the Validity Study Service, which may help ib 
explain the disparity in results. The correlation between 
meanhigh school gpa arid the multiple Rs are significant for 
R 2 . The standard deviation of high school rank is 
significantly related to R y but not to R 2 . In contrast, the 
standard deviation of high school grades is negatively 
related to both R 2 and /? 3 , but not significantly. 

The significant correlations between the multiple coeffi- 
cients predicting grades and institutional characteristics, 
using the institution as the unit of analysis, are shown in 
Table 2: A number of variables suggest that the more diverse 
the student body in nonacademic terms, the lower the 
multiple correlation. For example, undergraduate eriroll- 

eristics and Validity Coefficients 

fit R s N_ R^ 



152 


-.u 


174 


-.16* 


155 


-.16 


177 




97 


-.13 


115 


-.18* 


92 


-.07 


106 


-.15 


146 


.i7* 


167 


.20** 


150 


-.03 


170 


-.14* 


155 


-.11 


177 


-.08 


146 


-.27** 


177 


-.16* 


116 


-.15* 


133 


- .18* 


78 


-.20* 


90 


-.i7* 



& 



Tfable 3. Variables Associated with a Difference in Multiple Correlations of .03 or More 



Variqble_ 



Difference 



Difference 



College Type . 
Liberal Arts College 
University 
Women's College 
Cbec! Institution 
Catholic College 
Protestant College 



Location 
Metropolitan 
Large City 
Suburb- 
Small City 
Large Town 
Small Town 
Rural 



A c a de m i c P rog r a m s 

Acceleration Wfered 

Acceleration Not Offered 

Honors Offered 

Honors Not Offered 

Studcnt-Dcsigriod Major Allowed 

Not Allowed 

Double Major Allowed 

Not Allowed 

Individualized Study 

No Individualized Study 

Internships Available 

Not Available 

Year Abroad Available 

Not Available 

Tutoring Offered 
Not Offered 
Allows CLEP Credit 
Does Not Allow Credit 

Admissions Procedures 
Open Admissions Policy 
Not Open Policy 

Applications Accepted October-May 
Applications Accepted June -September 
No Closing Date 
Requires interview 

Does Not Require, b ut Recommends Inierv.cw 



285 
648 
43 
964 

146 
115 

157 
142 
231 
288 

62 
114 

13 

570 
438 
750 
258 
624 
384 
852 
156 
907 
101 

866 
142 
871 
137 
818 
190 
191 
817 

59 
949 
196 
332 
472 

50 
500 



.41 

.39 
.40 
.42 
;40 
.42 
.40 
.42 
.40 
.46 
.41 
.37 
.40 
.45 

.39 
.44 

.40 
.40 

.48 
.40 

.35 
.43 
.40 
40 
.40 



.00 
.01 
.01 

.08 



.02 
.02 
.02 
.02 
06 
.04 
.05 
.05 
.00 

.08 
.09 

.00 



87 
288 

20 
366 

64 

31 

51 
59 
64 
135 
23 
50 
5 

222 
165 

326 
61 

234 
153 

368 
19 

375 
12 

359 
28 

366 
21 

321 
66 
57 

330 



37 
350 

49 
118 
211 

-22 
228 



.55 
52 

:53 

.58 
.52 
.56 
.53 

:59 

.54 
.52 
.55 
.42 
.54 
.54 
.54 
.58 
.61 
.52 

.59 
.53 
.50 
.54 
.55 
.63 
.52 



.03 
.03 
.05 

.18 



.03 
.05 
.04 
.06 
.02 
.03 
.00 
.05 
.09 

.06 

.04 
.05 

.09 



merit is negatively related as are the percentage of students 
who are minority students (for /?0; the percentage of 
freshmen who are in financial need, and the percentage 
offered financial aid. These variables all suggest human 
diversity; 

Similarly, the greater the variety of activities available, 
especially sports activities, the lower the multiple correla- 
tion. This may be related to the overall diversity of the 
institutions' offerings. It may also reflect the number of 
distractions from academic studies. 

In contrast, the percentage of students who live in college 
housing is positively related to both R 2 arid R 3 . This may 
also represent a lower number of distractions from college 
life. 

The average R 2 arid R 3 figures for studies conducted in 
colleges of different types were also calculated: The 
categorical variables associated with differences in validity 
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study multiple correlations of .03 or more are shown in 
Table 3. The variables are grouped into clusters of related 
variables. The results suggest that grades were more 
predictable in studies conducted at liberal arts colleges than 
at universities, at coeducational institutions than at women's 
colleges, and at Protestant colleges than at Catholic 
colleges. It is easy to understand that universities have much 
more diverse programs than liberal arts colleges, arid that 
grades therefore are based on more kinds of courses and are 
therefore less predictable. The other differences are more 
difficult to understand. Catholic colleges include a number 
of large, urban institutions, whereas there are few Protestant 
institutions of that type; which may account for the 
difference: 

There is a large difference between the average Rs for 
studies conducted in colleges in metropolitan areas and 
those in small towns and rural areas. There seems to be a 

5 

0 



general, but hot perfect, negative association between the 
size of the /?s and the complexity of the communities 
surrounding the colleges. Presumably there are fewer 
commuting and part-time students in colleges in smaller 
settings. There are also probably fewer distractions From 
studies. 

A considerable number of variables involving the 
academic program were related to differences in the 
multiple correlations. In general, the more flexible ah 
institution, the lower the correlation. For example, studies 
conducted in colleges that offer honors programs, that allow 
the students to design their own majors, that allow double 
majors, and that offer tutoring resulted in lower Rs on the 
average; However, studies in colleges that offer acceleration 
and internships tend to obtain higher Rs. Studies in colleges 
that allow clep credits also have higher Rs: 

Admissions procedures are also related to the magnitude 
of the multiple correlation. Studies in colleges with open 
admissions policies tend to have higher Rs than do those in 
colleges that accept applications up to September or have no 
closing date. Additional analyses indicated that these 
college have lower average sat scores and admit a broader 
range o f students, as indicated by sat standard deviations. 
Studies in colleges that require interviews obtain higher Rs 
ihan studies in colleges that do hot require interviews. These 



results suggest that either heterogeneity of input or 
homogeneity of program are positively associated with the* 
predictability of grades. 

One of the interesting aspects of these results is the 
variety of variables that were hoi, related to predictability, 
although they might be expected to be on statistical or 
logical grounds.. These included the standard deviation of 
college grades, the minimum college gpa required for good 
standing, the percentage of freshmen in good standing, the 
percentage of bUt : bf^state students, the percentage of 
foreign students, and the percentage of graduates who go bh 
to graduate or professional school. This may be a credit to 
the robustness of the validity coefficient. 

Obviously, there is probably collinearity among these 
variables; For example, the finding that the validity 
coefficients are lower in studies in urban institutions than in 
institutions in small towns or rural areas may be due to a 
wider range of academic ability in the urban institutions. 
For these reasons, the multiple regression analyses 
described earlier were conducted. 

Multiple Regression Analyses 

In order to examine the relative contribution of the variables 
we have studied, we conducted multiple regression analyses 



Table 4. Prediction of R 2 and R 3 from Validity Study Data Using Four Grade Reporting Systems 




R, Results 



R, Results 



P 



Studies Using 

Student Reported Rank 
SAT-M— SD 
College GPA SD 
College GPA Mean 
SAT-M Mean 
N= 152. R= .62 

Studies Using 

Student Reported HS GPA 

HS GPA SD 

College GPA SD 

SAT-M SD 

S AT- V Mean 

HS GPA Mean 
N= 146, R = .49 

Studies Using 
College Reported Rank 

SAT-V SD 

SAT-M Mean 

College GPA SD 

SAT-M SD 

HS GPA SD 

HS GPA Mean 
N^379, R = .53 

Studies Using 

College Reported MS GPA 

SAT-V SD 

HS GPA SD 

SAT-V Mean 

College GPA 
N= 186, .57 



.31 
.44 
.42 
.31 



.44 
.46 
.14 
.37 
.25 



.34 
.34 
.19 
.25 
.14 
.14 



.44 
.22 
.26 
.20 



SAT-M— SD 
College GPA SD 
College GPA Mean 
HS GPA SD 

N= 152. R- .67 



College GPA SD 
HS GPA SD 
HS GPA Mean 
SAT-V Mean 
SAT-M SD 

N= 146 . R = .44 



SAT-M SD 

College GPA SD 
SAT-M Mean 
SAT-V SD 
HS GPA SD 
HS GPA Mean 
N-379. R ii .68 



SAT-V SD 
BS GPA SD 
SAT-V Mean 

N=I86, R = .57 



32 
:44 

.31 

:25 



-.48 
.33 
.45 

-.42 
.07 



.40 
-.29 
.22 
:12 
.10 
.08 



.42 
36 
.27 



Ta ble 5. Multiple Regre ssion Analyses for institutional Characteristics 





/?.; Results 


P 


R, Results 






N^5I3, R = .43 




N - 513, R = ;46 




All Studies and All Groups 


SAT Math Standard Deviation 
Number of Sports Activities 
Enrollment 


.39**+ 

-.06 

-;()7 


SAT Math Standard Deviation 
Enrollmern 

NUmber of Sports Activities 


.41*** 
-.13** 
.07 


Majors in Education 
Majors in Engineering 
Location 


- ;07 

.il** 


Majors in Education 
Majors in Engineering 
Location 


.11** 

-.04 
.15*** 




N = 36, R=.70 




N = 35. R= ;82 




Institutional Sample only 


S AT Math Standard De iatibh 
Number of Sports Activities 
Enrollment 


.46** 
-.35* 
.01 


SAT Math Standard Deviation 
Enrollment 

Number of Sports Activities 


46** 

.01 
-.27* 


Number of Female Applicants 
Percentage in College Housing 
Number In Financial Need 


-28 
.17 
.12 


Number of Female Applicants 
Percentage in College Housing 
Number in Financial Need 
Location 


-;65* 
.16 
13 

.24* 



♦This correlation is significant at the .05 level. 
**This correlation is significant at the .ui level. 
♦**This correlation is significant at the .001 level. 



Using R 2 and R> as the criteria. In the first set of analyses, the 
relative influence of the basic statistical prediction informa- 
tion was examined. In the second set, the role of 
institutional characteristics was also examined. 

Statistical data 

First, we examined the relationships between the standard 
statistical data and the size of the multiple correlations, 
using R 2 and R y as the dependent variables. The vss allows 
!° u L^j§*I school grade systems to be used: (1) student- 
reported high school rank, (2) student- reported high school 
gpa, (3) college-reported high school rank, and (4) college- 
reported high school gpa. Therefore, eight separate 
regression analyses were conducted. The results of the 
stepwise multiple regression analyses are shown in Table 4. 
Variables were selected until the multiple correlation no 
longer increased by at least .01. The positive predictor 
appearing most consistently across all analyses was the sat- 
hiathematical standard deviation, which occurred in six of 
the analyses; the sat- verbal standard deviation, which 
appeared in four of the analyses, was the next most 
consistent positive predictor. However, it is clear thai there 
is so much collinearity in the data that some other variables 
have erratic and surprising values in the equations. For 
example, the standard deviations of college grades had a 
positive value in two equations and a negative value in four. 
The standard deviation of high school grades had a positive 
value in three equations and a negative value in four These 
results are almost certainly due to the fact that several 
variables reflect the same underlying variable, the range of 
academic ability in the freshman class, and the method of 
analysis used — stepwise multiple regression. 



Analyses of institutional characteristics 
and statistical data 

Because there was so much collinearity among the measures 
of the variability of the academic ability of the students, 
only one was used in these analyses. This was the standard 
deviation of SAT : rnaiherhaiical scores, which was chosen 
because it had had the most consistent relationship to the 
size of the multiple correlation across the studies. The 
remaining institutional variables were chosen because they 
had the strongest relationships with the multiple cbrrela : 
tions, and also seemed to most adequately represent a 
domain. For example, total enrollment was chosen rather 
than undergraduate enrollment since it seemed more 
adequately to represent the diversity of programs arid people 
in an institution: The remaining variables chosen were: the 
number of sports activities, the percentage of students 
enrolled in education, the percentage of students enrolled in 
engineering, and the location of the institution (metropolis 
was high, rural location was low). 

The results for the institutional sample for all studies 
are shown in Table 5. In this case, the full multiple 
regression results are shown. (The interested reader can find 
the zero order correlations in other tables.) In the case of the 
institutional sample, the standard deviation of the sat- 
mathematical scores arid the number of sports activities had 
significant beta weights with the size of both multiple 
correlations predicting grades. (The Ns for the institutional 
sample were reduced considerably because The College 
Handbook file does not clearly distinguish between zeros 
and no data reported: Therefore, the number of cases that 
clearly had complete data was fairly small.) 

In the case of R 2 across all the studies, only sat- 
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Table 6. Correlations Between Statistical arid 
Institutional Characteristics arid the Difference 
Between the Correlations of Verbal arid Mathematical 
Scores with First-Year Grades ( r v - r m) 



Variable 


N 


Correlation 


SAT- Verbal Mean 


172 


-:02 


SAT-Mathematical Mean 


172 


.is* 


SAT- Verbal Standard Deviation 


172 


;08 


SAT-Mathematical Standard Deviation 


172 


-.04 


High School Grades 


80 


;22* 


Percent Minority Students 


165 


-.17* 


Percent Enrolled in Engineering 


44 


-.33* 



programs and thereby the meaning of grades (enrollment), 
the distractions from academic subjects, and the possible 
lower emphasis on scholarship (number of sports activities), 
and Se distractions within the community, as well as the 
number of commuting and part-time students (location). 

Analyses of the relative predictive effectiveness 
of Wand M 



*p<.05 

mathematical standard deviations arid location ("large city" 
was low, "rural" was high) were significantly related. For 
fl, across ail studies, these same variables were significantly 
related to the size of the multiple correlation, as well as to 
total enrollment, which had a negative weight, and to the 
percentage of students majoring in education, which had a 



positive 

Because of the variations allowed in the reporting of 

high school performance, multiple regressions were also 

prepared for each of the four grading systems across all 

studies. The results, shown in the Appendix, demonstrate 

that the variables most consistently related to the size of the 

multiple correlation coefficient were the standard deviation 

of the SAT-mathematical scores, total enrollment, the 

number of sports activities, and location. One interpretation 

of these results is that they show the influence of the 

variability of the academic ability of the incoming students 

(SAT-mathematical standard deviation), the variability of 

table 7. Correlations Between Statistical arid Institutional Characteristics and the Differences 
Between the Correlations of High School Grades and SAT Scores with First- Year College Grades 



Why does the sat- verbal score predict_co liege grades better 
in some colleges and the SAT-mathematical score predict 
better in others? To answer this question, we examined the 
differences in the correlations of sat- verbal and sat- 
mathematical scores with college grades. That is L we 
calculated the difference between T VC d FA and ?mcgp:a- Thei? 
we correlated this figure with the statistical and institutional 
characteristics of the colleges conducting the study. As 
shown in Table 6, SAT-verbal scores tended to predict better 
than SAT-mathematical scores in colleges that had higher 
mean mathematical scores and students with higher high 
school grades. The SAT-mathematical scores tended to be 
better predictors in colleges with higher proportions of 
minority students, and with large percentages of students 
enrolled in engineering. Interestingly, the differences in the 
power of V and M to predict grades were not related to the 
standard deviation of these scores. This is probably owing 
to the fact that most colleges tend to select students on both 
scores; thai is, the diversity of students would be about the 
same bri both measures . 

The tendency for verbal scores to be a better predictor 
at colleges with higher mathematical scores is hard to 
interpret. It may be, as Dawes (1975) has suggested, there is 



Corrections with: 



Variable 



Statistical Variables 
N in Study 
SAT-V Mean 
SAT-M Mean 
SAT-V Standard Deviation 
SAT-M Standard Deviation 
High School Rank Mean 
High School CPA Mean 
High School Rank Standard Deviation 
High School GPA Standard Deviation 

Institutional Variables 
Total Graduate Enrollment 
Percentage of Commuters 
Percentage of Minority Students 
Percentage Enrolled in Engineering 
Percentage Enrolled in Art 
Percentage Going bri to Graduate School 
Number of Students Offered Full Financial Aid 



N 


(r„-r v ) 






172 


-.21** 




.21** 


172 


.06 




.11 


172 


.08 




.11 


172 


- . 09 




.id 


!72 


-.08 




.07 


92 


.35** 




.36** 


80 


.03 




.16 


92 


-.21* 




.16 


80 


-;13 




.11 


112 


- .22 ' 




;20* 


68 


-.20 




.23* 


165 


-.11 




.17* 


44 


.21 




.04 


59 


-.14 




.17 


103 


.16 




.20* 


118 


-.i9 




.22* 



*p<;05 

**p<.01 
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table 8, Mean Statistical Characteristics of Institutions of Different Selectivity 



Selectivity by V and M 



Vanahte <SVO 507-900 901-1.000 > 1.000 



SAT : V Standard Deviation 


78 


87 


90 


89 


SAT-M Standard Deviation 


79 


91 


94 


87 


High School Rank Mean 


55 


56 


56 


63 


High School CPA Mean 


2.8 


2.9 


3.1 


3.2 


High School Rank Standard Deviation 


8.6 


7.3 


8.9 


6.4 


High School CPA Standard Deviation 


.52 


.56 


.51 


.51 


College CPA Mean 


2.2 


2.5 


2.6 


2.7 


College CPA Standard Deviation 


.8 


.8 


.7 


.6 


R, V arid M 


41 


.45 


.42 


.40 


R, V and M and H 


.50 


.56 


.57 


.51 


f v 


.34 


.39 


;37 


.33 


r M 


.37 


.39 


.35 


.32 


r H 


.38 


.47 


.48 


.42 


N 


19 


55 


57 


41 



Note: High School Rank is placed on a standard scale ranging from 20 to 80; High School GPA is on a scale from 0.0 to 4.0. 



compensatory selection which results in a restriction in 
range on mathematical scores, although the correlation with 
the SAT-mathematical standard deviation was not significant. 

There is a suggestion of an explanation in the related 
finding that studies conducted at colleges where higher high 
school grades are required also show verbal scores to be 
better predictors. It is possible that colleges that have 
traditional letters and science curricula tend to emphasize 
reading and writing skills that draw on skills reflected in the 
sat verbal scores. These colleges also tend to have higher 
academic standards. 

A similar explanation may apply to the results showing 
that colleges with high minority enrollments and with large 
proportions of engineering students tended to find sat- 
mathematical scores to be relatively better predictors. As 
documented elsewhere (Baird 1981), minority students tend 
to enroll in technical fields, and clearly, if curricula such as 
engineering and technical programs emphasize mathemati- 
cal skills, SAT-mathematical scores should be better 
predictors. 

Analyses of the relative predictive effectiveness 
of SAT scores arid the high school record 

Why are high school grades better predictors in some 
colleges and sat scores better predictors in others? To 
answer this question, we first calculated the difference 
between the correlation of high school grades or rank with 
college grades and the correlations of SAT-verbai and 
mathematical scores with college grades 
(tHsoPA CGPA — *v cgpa) and (t HS cpa ccpa - *m cgpaX We then corre- 
lated these differences with the statistical and institutional 
characteristics of the colleges in the validity studies. The 
results are shown in Table 7. Oddly, the idea that selection 
on one variable leads to a restriction of range on that 
variable and to a lower correlation did not seem to work. 
Although the mean high school rank is positively related to 



the correlations, that is, the high school rank is a better 
predictor in colleges with students ranking high in high 
school, the high school rank is a better predictor at colleges 
with a narrower range of high school ranks. 

The other variables provide some suggestions that the 
sat is a relatively better predictor at colleges with large 
graduate programs, relatively fewer commuters, minority 
students, arid students supported by financial aid. All this 
suggests thai high school grades are better predictors at 
diverse colleges or universities that do riot emphasize 
graduate study. However, high school grades tended to be 
better predictors at colleges that sent large proportions of 
students on to graduate school. 

Analyses of the influence of selectivity upon 
the pattern of validity results 

Colleges were grouped by their combined mean sat- verbal 
arid s_AT-rriatheiriati_cal scores. There were Four groups: less 
than 800, 800 to 900, 901 to 1,000, and greater than 1,000. 
The general results, shown in Table 8, indicated that the 
multiple corielatibris predicting college grades tended to be 
higher in the middle two categories and lower in the least 
and most selective institutions. Although it is sometimes 
contended that the range of ability should be smallest in the 
most selective institutions, this was not the case in this 
sample; in fact, sat scores were most truncated in the least 
selective institutions. Thus, the lower validity coefficients 
in the most selective institutions are probably due to other 
factors. One possibility is the relatively narrower range of 
college grades awarded to students in the most selective 
institutions, and the relatively narrower range of high school 
ranks represented among the freshmen. 




DISCUSSION 

These analyses show how the "validity" and relative 
predictive efficiency of sat scores are affected by many 
conditions — some statistical, some social, and some educa- 
tional . These conditions limit the size of the possible 
correlations and need to be considered when any correlation 
between the sat and grades is considered. Thus, the 
analyses indicate how the size of the correlation can be 
influenced by many Factors other than intrinsic validity of 
the scores as a measure of academic aptitude. The analyses 
of the correlates of the relative efficiency of sat- verbal and 
s\t : mathematical scores, these scores and the high school 
record, and the analyses by college selectivity support this 
finding. This study provides strong indications that grades 
are predicted more and less efficiently for understandable 
reasons related to the statistical and institutional characteris- 
tics of the colleges: 

The results of this study suggest that much could be 
learned by examining trie basic validity data that affect the 
use of the sat: The present analyses are fairly simple. 
Probably more could be understood by using analyses that 
are more statistically sophisticated. A current College 
Board project by Robert Boidt is, in fact, examining the 
question of validity generalization using very advanced 
methods. This project should lead to further insights. 

The results of the current study also suggest that to 
understand the prediction of grades we heed to pay attention 
to some fundamental considerations about the use of 
admissions tests, the characteristics of the students in any 
validity study, and the character of the educational 
experience in any specific college. Even a theoretically 
perfectly Valid measure of academic aptitude could not be 
expected to provide perfect or even very good prediction in 
certain circumstances. As argued by Weitzman (1982), the 
sat obviously provides highly useful information in 
situations where the prediction of academic performance is 
very difficult. In fact it may be those very situations where 
itr. contribution to the admissions prediction-decision 
situation is greatest. As the Taylor-Russell tables (1939) 
demonstrate, and as Cronbach and Glaser (1965) have 
pointed out 4 \ ... a test validity of .20 in one situation may be 
more beneficial than a test validity of .60 in another." 
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Appendix: Regression Results for Institutional and Statistical Studies for Variables Using Different 
Grading Systems 





R, Results 


P 


R;, Results 




Studies 
Using 

College-Reported 


N = 235, R = .58 


49*** 

.11 
= .2i** 
-.09 
-.14** 

.09 


N = 235, R = .62 








SAT Math standard deviation 
Number of Sports Activities 
Enrollment 
Majors in Education 


SAT Math standard deviation 
Enroll merit 

Number of Sports Activities 
Majors in Education 


.51*** 
-:28*** 

13* 
-:01 
-.06 

.13* 


Rank 


Majors in Engineering 
Location 


Majors in Engineering 
Location 


Studies 
Using _ 
Student -Re ported 
Rank 


N = 63, R- .43 




N = 63, R= .61 








SAT Math. standard deviation 

Number of Sports Activities 

Enrollment 

Majors in Education 

Majors in Engineering 

Location 


.29** 
.13 
- .32** 

:07 
-15 
-:05 


SAT Math standard deviation 
Enrollment 

Number of Sports Activities 
Majors in Education 
Majors in Engineering 
Location 


■47*** 
-:39** 

.32* 
-:04 
-.06 

.07 


Studies 
Using 

ColIege : Re ported 
GPAs 


N^il4, R = .6I 


.41 *** 
-.51*** 
■ 35** 

:I8* 

-♦|5_ 
: 19* 


N « 114, R==.56 


■34*** 

:26 

-.48*** 

.06 
-.16 

.22** 






SAT Math. Standard deviation 
Number of Sports Activities 
EnrpllmenL _ 
Majors in Education 
Major? in Engineering 
Location 


SAT Math standard deviation 
Enrollment 

Number of Sports Activities 
Majors in Education 
Majors in Engineering 
Location 




N« ioi, R- .31 


Q5 
-.24 
- 01 

:I0 
.26 
-;04 


N= 101, R==.30 


Q2 

-:26 

.09 
:08 
.24 
.01 


Studies 
Using 

S tit dent- Reported 
GPAs 


SAT Mathsjandard deviation 
Number of Sports Activities 
Enrollment, _ 
Majors in Education 
Majors in Engineering 
Location 


SAT Math standard deviation 
Enrollment 

Number of Sports Activities 
Majors in Education 
Majors in Engineering 
Location 



♦This correlation is significant at the .05 level. 
♦♦This correlation is significant at the :0I level. 
♦♦♦This correlation is significant at the .001 level. 
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